Input/output port rotation in a storage area network device

ABSTRACT

In one aspect of the present description, in an input/output (I/O) device having multiple CPUs and multiple I/O ports, a cycle of I/O port rotations is initiated in which each port rotation of the cycle includes rotating an assignment of at least one I/O port from one CPU to a different CPU of a plurality of the CPUs. In the illustrated embodiment, an I/O port assignment for each CPU of the plurality CPUs is rotated for at least a portion of the cycle. Other features and aspects may be realized, depending upon the particular application.

BACKGROUND

1. Field

The present description relates to a method, system, and computerprogram for assigning input/output ports in a storage area network.

2. Description of Related Art

A storage area network (SAN) is frequently used to couple remotecomputer storage devices such as disk arrays, tape libraries, opticaljukeboxes or other storage devices, to hosts in a manner which permitsthe storage devices to appear to the operating systems of the hosts aslocally attached to the hosts. Fibre Channel (FC) is a high speednetworking technology in which signals may be transmitted over varioustransmission media including fiber optic cable or twisted pair coppercables, for example.

A storage controller may have multiple servers which are assignedinput/output (I/O) tasks by the hosts. The I/O tasks may be directed tospecific volumes in the storage. The storage controller may further havemultiple input/output (I/O) adapters such as host adapters which enablethe servers to communicate with the hosts, and device adapters whichenable the servers of the storage controller to communicate with thestorage devices. An I/O adapter may have multiple CPUs which receiveinput/output requests from one or more servers of the storage controllerto perform input/output operations in which data is read from or writtento storage through various I/O ports of the I/O adapter. I/O adaptersfrequently have a shared resource environment in which resources, suchas hardware resources such as I/O ports, and memory resources such asdata structures in memory, for example, may be shared by multiple CPUs.An example of a shared resource environment used in I/O adapters is theSymmetric Multiprocessing (SMP) environment, for example.

Access to the shared resources may be controlled using appropriate rulessuch as single ownership or shared ownership rules, for example. In onetypical shared resource environment, access to a shared resource may beaccomplished by using lock words in shared memory data structures thatcontrol access to the resource being accessed. Another method known inthe art is to group associated data structures for various resourcestogether in a task control block that is allocated each time a taskbegins and is locked and unlocked by each CPU as operations of the taskare carried out.

In a typical shared resource environment, each of the I/O ports willhave its work performed almost entirely under the control of a singleCPU until the tasks of that I/O port are completed. Such an arrangementis believed to improve the performance of a multi-CPU adapter by, forexample, reducing cache thrashing.

A list of the I/O port or ports assigned to a particular CPU may bestored in an element of a data structure for that CPU. The particularCPU assigned to a particular I/O port may also be identified by anappropriate CPU owner ID stored in an element of a data structureassigned to the particular I/O port.

The CPU assigned a particular I/O port may prevent other CPUs fromaccessing that particular I/O port by setting a lock in an element of adata structure assigned to that particular I/O port. Once the I/O portis no longer owned by a particular CPU, the lock element may be cleared.

The I/O tasks to be performed in connection with a particular I/O portmay be identified in an element of a data structure assigned to aparticular I/O port. The I/O work assignments performed in connectionwith an I/O port include, for example, communicating with the particularcommunication protocol hardware, such as Fibre Channel Optics hardware,for example, submitting work to a direct memory access (DMA) engine inlogic hardware such as an ASIC, for example, in the I/O adapter to movedata in and out of one or more of the caches, etc.

SUMMARY

Processes, hardware, software, firmware and combinations thereof mayperform operations including assigning input/output (I/O) ports to aplurality of central processing units (CPUs) of an I/O adapter and, foreach CPU, and transferring data between the CPU and each I/O portassigned to that CPU. In one aspect of the present description, theaforementioned operations may be combined with cycles of I/O portrotations wherein each port rotation of a cycle includes rotating anassignment of at least one I/O port from one CPU to a different CPU ofthe plurality of CPUs. Furthermore, an I/O port assignment for each CPUof the plurality CPUs may be rotated for at least a portion of thecycle.

In another aspect, the data transferring may include initiating a taskof transfer of data at the at least one I/O port under control of theone CPU and continuing the task of transfer of data at the at least oneI/O port under control of the different CPU following the rotation ofassignment of the at least one I/O port from the one CPU to thedifferent CPU. Also, determining whether to initiate a cycle of I/O portrotations, may include determining whether the number of CPUs of theplurality of CPUs is greater than or equal to the number of I/O ports ofthe I/O adapter.

In another aspect, determining whether to initiate a cycle of I/O portrotations, may include determining whether a predetermined time periodhas expired. In addition, initiating a cycle of I/O port rotations mayinclude setting by one CPU, a rotation flag for the remaining CPUs ofthe plurality of CPUs, to initiate port rotations for the remaining CPUsof the plurality of CPUs.

In still another aspect, each port rotation of the cycle of I/O portrotations may include determining for each CPU, whether to rotate a portassignment of the particular CPU to a different CPU of the plurality ofCPUs. Furthermore, initiating a cycle of I/O port rotations may includesetting by one CPU, a rotation flag for the remaining CPUs of theplurality of CPUs, to initiate port rotations for the remaining CPUs ofthe plurality of CPUs, and determining for each CPU whether to rotate aport assignment, may include checking whether the rotation flag has beenset for that particular CPU.

In a further aspect, determining for each CPU whether to rotate a portassignment, may include determining whether that particular CPU has anyactive I/O ports. Also, determining for each CPU whether to rotate aport assignment, may include determining whether that CPU has feweractive I/O ports than any other CPU of the plurality of CPUs.

In yet another aspect, determining for each CPU whether to rotate a portassignment, may include selecting an I/O port owned by the particularCPU for rotation of its port assignment to another CPU. Also, selectingan I/O port owned by the particular CPU for rotation of its portassignment to another CPU, may include selecting the active I/O portwhich has been owned the longest by the particular CPU.

In still another aspect, the CPUs of the plurality of CPUs may besequentially numbered in a sequence and rotating an assignment of atleast one I/O port from one CPU to a different CPU of the plurality ofCPUs may include rotating an assignment of at least one I/O port fromone CPU to the next sequentially numbered CPU in the sequence of thesequentially numbered CPUs.

Other features and aspects may be realized, depending upon theparticular application.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an embodiment of a computing environment in which anaspect of the present description may be employed.

FIG. 2 illustrates an embodiment of an I/O adapter in which I/O portassignments are rotated in accordance with the present description.

FIGS. 3a-3c depict examples of data structures for the I/O adapter ofFIG. 2.

FIGS. 4 and 4 a depict examples of operations to perform I/O portassignment rotations in accordance with one aspect of the presentdescription.

FIG. 5 depicts a chart illustrating one example of I/O port assignmentrotations in accordance with one aspect of the present description.

FIG. 6 depicts another example of operations to perform I/O portassignment rotations in accordance with one aspect of the presentdescription.

FIG. 7 depicts another chart illustrating another example of I/O portassignment rotations in accordance with one aspect of the presentdescription.

DETAILED DESCRIPTION

FIG. 1 illustrates one example of a computing environment employing anaspect of the present description. In accordance with the presentdescription, in an input/output (I/O) device having multiple CPUs andmultiple I/O ports, a cycle of I/O port rotations is initiated. Eachport rotation of the cycle includes rotating an assignment of at leastone I/O port from one CPU to a different CPU of a plurality of the CPUs.In the illustrated embodiment, an I/O port assignment for each CPU ofthe plurality CPUs is rotated for at least a portion of the cycle. Suchan arrangement is believed to improve workload balancing among the I/Oports, for example. However, other features or aspects may be realized,depending upon the particular application.

In the environment of FIG. 1, one or more hosts 122 communicateInput/Output (I/O) tasks or other processing requests directed to astorage 124, through a storage area network 126 which manages access tothe storage 124. In one embodiment, the storage area network 126includes a plurality of servers 128 a, 128 b, 128 c, 128 d, eachincluding a processor and a memory. Each processor can include one ormore central processing units (CPUs) available as processing resourcesto the associated server 128 a-128 d. Although the storage area network126 is depicted as including four servers 128 a-128 d, for purposes ofillustration, it is appreciated that the number of servers may begreater or lesser, depending upon the particular application.

Each server 128 a-128 d may have one or more logical partitionsexecuting in the server 128 a-128 d. Each logical partition separatelyexecutes an operating system and device drivers. The logical partitionscomprise a division of the processors into logically independent butcooperating processing systems each having their own operating systemsand device drivers. Multiple logical partitions may execute in eachserver 128 a-128 d, managed by a supervisor module for that server 128a-128 d.

Each device driver provides an interface between the operating system inthe logical partition in which the device driver executes, and a device,such as an I/O adapter including host adapters 132 a, 132 b . . . 132 n,and device adapters 144 a, 144 b . . . 144 n. The host adapters 132 a,132 b . . . 132 n, enable the servers 128 a-128 d to communicate withthe hosts 122. The device adapters 144 a, 144 b . . . 144 n enable theservers 128 a-128 d to communicate with the storage 124. Thus, theservers 128 a-128 d share devices, such as adapters 132 a, 132 b . . .132 n, 144 a, 144 b . . . 144 n. The variable “n” is used to denote aninteger instance of an element, and may indicate different or the sameinteger value when used with different elements. For instance, 132 n,144 n, may indicate a same or different number of host adapters 132 n,and device adapters 144 n.

The servers 128 a-128 d communicate with the host adapters 132 a, 132 b. . . 132 n, and the device adapters 144 a, 144 b . . . 144 n, over afabric 150. The fabric 150 may comprise one or more interfaces providingcommunication paths between the servers 128 a-128 d and adapters. A pathcomprises the hardware in the fabric 150 that enables communication withshared adapters over the fabric. The fabric may comprise for example, aFibre Channel arbitrated loop configuration, a serial loop architectureor a bus interface, such as a Peripheral Component Interconnect (PCI)interface such as a PCI-Express interface. Each server 128 a-128 d maybe assigned a portion of the adapters 132 a, 132 b . . . 132 n, 144 a,144 b . . . 144 n during initialization.

In the example of FIG. 1, the fabric 150 may include a switch 160 whichfacilitates sharing of the adapters 132 a, 132 b . . . 132 n, 144 a, 144b . . . 144 n by each of the servers 128 a-128 d. The servers 128 a-128d may be assigned to handle I/O requests directed to specific volumesconfigured in the storage 124. The servers 128 a-128 d communicate overconnections 168 enabling processor inter-communication to manageconfiguring operations performed with respect to the shared devices,such as the shared adapters 132 a, 132 b . . . 132 n, 144 a, 144 b . . .144 n.

FIG. 2 shows an I/O adapter 200 which may be the same as or similar tothe adapters 132 a, 132 b . . . 132 n, 144 a, 144 b . . . 144 n ofFIG. 1. In the illustrated embodiment, the I/O adapter 200 may be aFibre Channel adapter, for example. The I/O adapter 200 may utilizeother communication protocols, depending upon the particularapplication.

The adapter 200 has a plurality of CPUs, that is, CPU0, CPU1, CPU2 . . .CPUn which receive input/output requests from one or more servers 128a-128 d, to perform input/output operations in which data is read fromor written to storage through various I/O ports P0, P1, P2, . . . Pn. Asused herein, the term CPU refers to a processing resource which may beimplemented with one or more physical processing cores which may beapportioned into one or more processing resources associated with eachprocessing core. Thus, as used herein, the term CPU may be used to referto a one or more physical processing units or cores or a portion of suchproviding a processing resource.

The adapter 200 has additional resources such as I/O port resourcesprovided by the I/O ports P0, P1, P2, . . . Pn. Thus, as used herein,the term I/O port may be used to refer to a physical I/O port or aportion of such providing an I/O port resource to one or more CPUs.

Additional resources of the adapter 200 include memory resourcesprovided by a memory 210. These memory resources include caches 212which may temporarily store data such as I/O data for the CPUs, CPU0,CPU1, CPU2 . . . CPUn. Each CPU, CPU0, CPU1, CPU2 . . . CPUn, may haveaccess to one or more caches of the adapter 200.

The memory resources further include programming resources 214 which mayinclude operating systems, programs and device drivers, for example, forthe CPUs, CPU0, CPU1, CPU2 . . . CPUn. Other memory resources includedata structures 216. Each CPU may have access to one or more datastructures 216.

FIG. 3a shows an example of private data structure 220 for a particularCPU. Thus, read or write access to the CPU private data structure 220may be limited to a particular CPU assigned to that particular CPUprivate data structure 220. Each of the CPUs, CPU0, CPU1, CPU2 . . .CPUn, may similarly have a CPU private data structure 220.

FIG. 3b shows an example of a CPU shared data structure 224. In theillustrated embodiment, and as explained in greater detail below, theCPU shared data structure 224 includes an element 226 in which a “ForceRotation Flag” may be stored by one CPU to initiate I/O port assignmentrotation by the CPU assigned the particular CPU shared data structure224. Thus, read or write access to the CPU shared data structure 224 maybe permitted for more than one CPU.

Each I/O port may similarly have one or more data structures associatedwith a particular I/O port. FIG. 3c shows an example of an I/O port datastructure 230 for a particular I/O port. Each I/O port may be controlledby a particular CPU, CPU0, CPU1, CPU2 . . . CPUn. Thus, read or writeaccess to the I/O port data structure 230 may be limited to theparticular CPU associated with the particular I/O port.

In the illustrated embodiment, the adapter 200 is a shared resourceenvironment. One example of such as a shared resource environment is theSymmetric Multiprocessing (SMP) environment, for example. Other types ofshared resource environments may be suitable, depending upon theparticular application.

In a shared resource environment, resources such as hardware resourcesincluding I/O ports, and memory resources including data structures inmemory, for example, may be shared by multiple CPUs. Accordingly, accessto the shared resources may be appropriately controlled usingappropriate rules such as single ownership or shared ownership rules,for example. In one typical shared resource environment, access to ashared resource may be accomplished by using lock words in shared memorydata structures that control access to a particular resource. Anothermethod known in the art is to group associated data structures togetherin a task control block that is allocated each time a task begins and islocked and unlocked by each CPU as operations of the task are carriedout.

As previously mentioned, the I/O adapter 200 has a plurality of I/Oports P0, P1, P2, . . . Pn which are serviced by one or more CPUs, CPU0,CPU1, CPU2 . . . CPUn. In the illustrated embodiment, the I/O port datastructure 230 for the associated I/O port P0, P1, P2, . . . Pn, haselements duplicated for each of the I/O ports and are indexed by portnumber. Thus, if the I/O port data structure 230 of FIG. 3c is for theI/O port P0, for example, the I/O port data structure 230 has an element240 which stores the port index number or other identification for theport P0. The particular I/O port associated with each of the other I/Oport data structures 230 for the remaining I/O ports P1, P2, . . . Pn issimilarly identified in an element 240 with the port index number forthe particular port number.

Such a common layout for the I/O port data structures 230 for the I/Oport ports P0, P1, P2, . . . Pn can facilitate application of commoncode and structures on adapters that deploy different numbers of ports(e.g. 1, 2, 4, 8, etc). It is appreciated that the layout for differentI/O port data structures for different I/O ports, may vary, from I/Oport to I/O port, depending upon the particular application.

In the illustrated embodiment, each of the I/O ports will have its workperformed almost entirely by a single CPU at a time. Such an arrangementis believed to improve the performance of a multi-CPU adapter. Theparticular CPU assigned to a particular I/O port may be identified by a“CPU owner ID” or other suitable identification stored in an element 242of the I/O port data structure 230 assigned to the particular I/O port.

A list of the I/O port or ports assigned to a particular CPU may bestored in an element 250 of the CPU shared data structure 224 for thatCPU. The CPU assigned a particular I/O port may prevent other CPUs fromaccessing that particular I/O port by setting a lock in an element 260of the I/O port data structure 230 assigned to that particular I/O port.Once the I/O port is no longer owned by a particular CPU, the lockelement 260 may be cleared.

The I/O tasks to be performed in connection with a particular I/O portmay be identified in a “work assignments” element 244 of the I/O portdata structure 230 assigned to a particular I/O port. The I/O workassignments performed in connection with an I/O port includes, forexample, communicating with the particular communication protocolhardware, such as Fibre Channel Optics hardware, for example, submittingwork to a direct memory access (DMA) engine in logic hardware such as anASIC, for example, in the adapter 200 to move data in and out of one ormore of the caches 212, etc. It is believed that assigning a particularI/O port to a particular CPU helps reduce the amount of cache thrashingthat can occur when multiple CPUs access a port or its data structures.Other work assignments may be assigned to an I/O port, depending uponthe particular application.

In an I/O adapter with multiple CPUs and multiple ports, it's possiblethat at a particular time, the number of ports actively running I/Otasks may not be evenly distributable among all of the CPUs. Forexample, an adapter may have two CPUs and four I/O ports. However due tovarious factors, it is possible that only three of the four I/O ports isactively performing work. Accordingly, if the three active I/O ports aredistributed between the two CPUS, one CPU will be assigned two activeI/O ports whereas the other CPU will be assigned only one active I/Oport. This unbalanced port ownership can lead to less work beingperformed by each of the ports that are sharing a CPU, and more workbeing performed for the port that has a single CPU assigned to it. As aresult, an even distribution of workload amongst the active I/O portsmay be impaired.

In accordance with one aspect of the present description, ownership orassignments of one or more active I/O ports may be rotated among CPUs ofthe I/O adapter. It is believed that such an arrangement may improve thebalance of the workload among the active I/O ports, for example.However, it is appreciated that in other applications, other featuresmay be realized in addition thereto or instead of an improvement inworkload balance, depending upon the particular application.

Thus, in one application of the present description, rotation of I/Oport assignments may be applied in instances in which the number ofactive ports is not evenly distributable among all of the CPUs in theadapter. To improve the balance of the work that is performed for allactive ports in an I/O adapter, port ownership of some or all of the I/Oports owned by the CPUs may be rotated in one or more cycles of portrotations. Once it is determined that port rotation should occur, eachCPU looks for an I/O port among the I/O ports owned by that CPU, whichit deems suitable for rotation, and if found, rotates it to another CPU.

For example, a timer may be used to initiate a cycle of port rotationsand each port rotation within the cycle. It is appreciated that othercriteria may be used to determine when to initiate a port rotation or acycle of port rotations, depending upon the particular application.

In addition, to determine a suitable port for port rotation, a CPU may,in one embodiment, determine how many active ports it currently owns inrelation to the other CPUs in the adapter. If the CPU finds that itcurrently has fewer ports than any other CPU, in one embodiment, it willnot rotate an active port that it owns. However, if a CPU determinesthat it has at least the same number of ports as all the other CPUs, ormore ports than all other CPUs, it will rotate the active port it hasowned the longest. It is appreciated that other criteria may be used todetermine a suitable I/O port for assignment rotation, depending uponthe particular application.

Thus, once the timer has indicated that a cycle of port rotations shouldoccur, over the course of that cycle, each CPU owning an appropriatenumber of I/O ports, finds the active port it has owned the longest androtates it to the next n+1 CPU in sequential order. In one embodiment,the cycle of port rotations is complete once an I/O assignment for eachCPU of the I/O adapter is rotated for at least a portion of the cycle.The next time the timer expires following completion of a cycle, anothercycle of rotations may be repeated.

It is believed that such an arrangement can improve distribution ofownership of I/O ports amongst the CPUs of the adapter and thus improvedistribution of workload amongst the CPUs of the adapter. It isappreciated that other features may be realized in addition thereto orinstead of workload balancing, depending upon the particularapplication.

FIG. 4 depicts an example of operations of an I/O adapter in whichassignments of one or more active I/O ports may be rotated among CPUs ofthe I/O adapter. In a first operation, I/O ports are assigned (block310) to a plurality of CPUs. Following each assignment (or reassignmentdescribed below), the CPUs conduct I/O operations (block 320, FIG. 4)through their assigned I/O ports. A determination (block 330, FIG. 4) ismade as to whether a cycle of I/O port rotations should be initiated. Ifso, assignments of I/O ports may be rotated (block 340, FIG. 4) amongCPUs of the I/O adapter.

FIG. 4a shows a more detailed example of operations 340 for performing acycle of port rotations once a cycle of port rotations has beeninitiated (block 330, FIG. 4). In a first operation of the cycle, a portrotation is performed (block 350). FIG. 5 depicts an example of I/Oports being assigned and then reassigned to a plurality of CPUs in asequence of port rotations PR0, PR1, PR2 . . . PRn. In this example, theI/O adapter has four CPUs designated CPU0, CPU1, CPU2 and CPU3. Inaddition, the I/O adapter has eight I/O ports P0-P7, in which in thisexample, I/O ports P0-P4 are actively running I/O tasks and ports P5-P7are idle.

In the assignments resulting from port rotation PR0 of FIG. 5, I/O portP0 is assigned to CPU0, I/O port P1 is assigned to CPU1, I/O ports P2,P3 are assigned to CPU2, and I/O port P4 is assigned to CPU3. Followingeach assignment (or reassignment described below), the CPUs conduct I/Ooperations (block 360, FIG. 4a ) through their assigned I/O ports. Thus,the assignments of port rotation PR0, CPU0 transfers data through itsassigned port P0, CPU1 transfers data through its assigned port P1, CPU2transfers data through its assigned ports P2, P3, and CPU3 transfersdata through its assigned port P4.

A determination (block 370, FIG. 4a ) is made as to whether the cycle ofI/O port rotations has been completed. If so, the port rotations may beterminated (block 380) until another cycle is initiated. If not, anotherrotation of I/O ports may be performed (block 350, FIG. 4a ) among CPUsof the I/O adapter. In the example of FIG. 5 the next port rotationafter port rotation PR0 is designated port rotation PR1. Thus, in thecourse of port rotation PR1, the assignment of I/O port P2 is rotated toa different CPU which is CPU3 in this example.

To effect this change of ownership, the CPU2 performing the transfer canupdate the list of ports owned in element 250 (FIG. 3b ) of the CPUshared data structure 224 for the CPU2 to indicate that I/O port P2 isno longer owned by CPU2. In addition, the CPU2 can change the CPU ownerID stored in element 242 (FIG. 3c ) of the I/O port data structure 230assigned to I/O port P2 from CPU2 to CPU3 and set the lock 260 toinhibit access to I/O port P2 by CPUs other than CPU3. In addition, theCPU2 can update the list of ports owned in element 250 (FIG. 3b ) of theCPU shared data structure 224 for the CPU3 to indicate that I/O port P2is now owned by CPU3.

In the course of rotating the assignment of I/O port P2 from CPU2 toCPU3, the I/O data transfer operations of I/O port P2 will switch frombeing under the control of CPU2 to being under the control of the CPU3.Thus, once the assignment of I/O port P2 is rotated by CPU2 to CPU3 asdescribed above, any of the I/O data transfer tasks of I/O port P2initiated or otherwise performed by CPU 2 continue but under the controlof CPU3 instead of CPU2. CPU3 will continue to carry out those tasks forI/O port P2 so long as CPU3 owns I/O port P2 or until the tasks arecompleted under the ownership of CPU3. It is appreciated that in someapplications, particular I/O data transfer operations of a task may beinterrupted to allow for ownership transfer and then resumed once theownership transfer of the I/O port has been accomplished.

Thus, following this port rotation PR1, the CPUs conduct I/O operations(block 360, FIG. 4a ) through their assigned I/O ports. Thus, CPU0continues to transfer data through its assigned port P0, CPU1 continuesto transfer data through its assigned port P1, CPU2 continues totransfer data through its assigned port P3, and CPU3 continues totransfer data through its assigned port P4 and CPU3 begins to transferdata through its newly assigned I/O port P2.

On the next port rotation (block 370, FIG. 4a ), the assignments of oneor more I/O ports may again be rotated (block 350, FIG. 4a ), from oneor more CPUs to one or more different CPUs. In the example of FIG. 5 thenext port rotation is designated port rotation PR2. Thus, in the courseof port rotation PR2, the assignment of I/O port P4 is rotated to adifferent CPU which is CPU0 in this example.

In the example of FIG. 5, there are four CPUs and five active I/O portssuch that at any one time, one CPU has two I/O ports and the remainingCPUs have one I/O port. The I/O port assignments are rotated in a cycleof port rotations such that, in the illustrated embodiment of FIG. 5,each CPU has a second assigned I/O port for at least a portion of thecycle. Thus, in the illustrated embodiment of FIG. 5, a cycle of portrotations includes four sequential port rotations. It is appreciatedthat the number of port rotations in a cycle may vary depending upon thenumber of CPUs, the number of I/O ports, and the manner in which I/Oports are rotated. For example, in some rotation sequences, some CPUsmay be selected to have more I/O ports more frequently than other CPUsover the course of a cycle. Other rotation sequences may be selected,depending upon the particular application.

In another aspect of the present description, the cycles of I/O portrotations may be spaced using a timer, for example. In otherapplications, the elapsed time between cycles may be a function of otherevents.

In another aspect of the present description, a port may be selected forassignment rotation as a function of how long it has been owned by aparticular CPU. Thus, the I/O port that has been owned the longest byCPU2 at the time of port rotation PR1 may be selected for assignmentrotation. In the example of FIG. 5, I/O port P2 was owned by CPU2 longerthan CPU2 owned I/O port P3. Accordingly, I/O port P2 was reassigned toanother CPU in the course of port rotation PR1. However, it isappreciated that other criterion may be used to select a port forassignment rotation to another CPU.

In yet another aspect of the present description, the CPU to which theassignment of an I/O port is rotated, may be selected in a linearsequential fashion. For example, an I/O port being rotated from a CPUn,may be reassigned to the next CPU in sequence, that is CPUn+1, forexample. An I/O port being rotated from the highest numbered CPUn, maybe reassigned to the first CPU in sequence, that is CPU0 for example.Thus, the rotation sequence based on CPU numbers in this example wouldbe 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, etc. Accordingly, in the example ofFIG. 5, an I/O port which was owned by CPU2, is reassigned to CPU3, thenext CPU in sequence by CPU number. Similarly, an I/O port which wasowned by CPU3, is reassigned to CPU0, the next CPU in sequence by CPUnumber. It is appreciated that other sequences may be used and othercriteria may be used to select the destination CPU of an I/O portreassignment.

In the illustrated embodiment of FIG. 5, the port rotations PR0, PR1,PR2 . . . PRn, are each initiated by the expiration of a timer such thatinitiations of each of the port rotations may be relatively evenlyspaced in time. However, it is appreciated that in other applications,the duration of time between adjacent port rotations may vary, dependingupon circumstances or the criteria used to determine when to initiate aport rotation or a port rotation cycle.

As previously mentioned, a port may be selected for assignment rotationas a function of how long it has been owned by a particular CPU. Thus,the I/O port that has been owned the longest by CPU3 following portrotation PR1 may be selected for assignment rotation in the next portrotation PR2. In the example of FIG. 5, I/O port P4 was owned by CPU3longer than CPU3 owned I/O port P2 since I/O port P2 was just reassignedto CPU3 in the prior port rotation (PR1). Accordingly, I/O port P4 isreassigned by port rotation PR2 to the next CPU in sequence, that isCPU0, the next CPU in sequence by CPU number.

Following this port rotation PR2, the CPUs conduct I/O operations (block360, FIG. 4a ) through their assigned I/O ports. Thus, CPU0 transfersdata through its assigned ports P0, P4, CPU1 transfers data through itsassigned port P1, CPU2 transfers data through its assigned port P3, andCPU3 transfers data through its assigned port P2. Port rotations PR3-PR5operate in a similar fashion.

In the illustrated embodiment, each of the CPUs, CPU0, CPU1, CPU2 andCPU3, performs a scan loop in which the CPU scans various items fortasks to perform. Once the scan loop is completed, the CPU returns tothe top of the loop and repeats. FIG. 6 shows a task 400 of a scan loopwhich provides a more detailed example of operations for rotating I/Oport assignments in accordance with one aspect of the presentdescription.

In one operation, upon entering the task 400 (block 402) the CPUexecuting the task 400 of FIG. 6 determines (block 410) whether thenumber of CPUs in the I/O adapter is greater than or equal to the numberof I/O ports in the I/O adapter. The number of CPUs and I/O ports in theadapter may be determined by examining the CPU shared data structure 224and I/O port data structure 230 of the CPUs and I/O ports, respectively,for example. In accordance with one aspect of the present description,it is recognized that in some applications, I/O port rotation may bebypassed (block 412) if the number of CPUs is greater than or equal tothe number of I/O ports in the adapter. In such situations, each CPUwill only own at most one port in the adapter. However, it isappreciated that in other applications, I/O port rotation may bebeneficial even in instances where the number of CPUs is greater than orequal to the number of ports in the adapter.

If the number of CPUs of the adapter is less than the number of I/Oports, a determination (block 415) is made as to whether the CPUexecuting the task 400 is

CPU0. In the illustrated embodiment, one CPU, such as CPU0 is selectedto determine (block 420) whether to initiate I/O port rotations. If so,as explained in greater detail below, the CPU0 sets a flag for each ofthe other CPUs, CPU1, CPU2, CPU3, to initiate an I/O port rotation forthe other CPUs if appropriate. It is appreciated that, in otherapplications, any or all of the CPUs may be selected to determinewhether port rotation is to occur and if so to initiate I/O portrotation for the other CPUs.

As previously mentioned, a cycle of rotations of I/O port assignmentsmay occur in a sequence of events or cycles. In the illustratedembodiment, the initiation of cycles may be evenly spaced using a timer,for example. In other applications, the elapsed time between cycleinitiations may vary. For example, each time a CPU completes a scanloop, a counter may be incremented. Once the counter reaches apredetermined threshold, it may be determined to be an appropriate timeto initiate a cycle of I/O port rotations or to initiate a port rotationwithin a cycle of port rotations. Frequently the time to execute a scanloop may depend upon the number and complexity of the tasks to becompleted in a particular scan loop. Hence, the duration of time betweenport rotations or cycles of I/O port assignment rotations may vary fromcycle to cycle or port rotation to port rotation. Other techniques maybe used to identify an appropriate time to execute a port rotation or acycle of I/O port rotations, depending upon the particular application.Should CPU0 determine (block 420) that it is not an appropriate time toinitiate I/O port assignment rotations, the port rotation may bebypassed (block 412).

If a determination (block 415) is made that the CPU executing the task400 is a CPU other than CPU0, a flag (e.g. Force Rotation Flag stored inelement 226, FIG. 3b ) is examined (block 422) to determine if an I/Oport rotation should be executed by that CPU. As previously mentioned,the I/O port rotation flag is set by CPU0 in each of the CPU shared datastructure 224 assigned to each of the remaining CPUs, CPU1, CPU2, andCPU3. Should it be determined (block 422) by a CPU other than CPU0 thatthe Force Rotation Flag has not been set, the port rotation may bebypassed (block 412).

Conversely, should it be determined (block 422) by a CPU other than CPU0that the Force Rotation Flag has been set for that CPU, or should CPU0determine (block 420) that it is an appropriate time for an I/O portassignment rotation, a determination (block 425) may be made as towhether the particular CPU executing the task 400 has any active ports.Lacking any active ports, the CPU executing the task 400 may bypass I/Oport selection and rotation (blocks 430-440).

If the particular CPU executing the task 400 does have active ports(block 425), the CPU executing the task 400 may determine (block 430)which of the I/O ports owned by that CPU has been owned the longest. Inaddition, the CPU executing the task 400 determines (block 435) if theparticular CPU has fewer active I/O ports than any other CPU, and if so,the CPU executing the task 400 may bypass port rotation (block 440). Thenumber of I/O ports owned by the other CPUs may be determined byexamining the CPU shared data structure 224 (FIG. 3b ) for the otherCPUs, for example.

Thus, in the example of FIG. 5, as a result of port rotation PR0, it isindicated that CPU0, CPU1 and CPU3 each has fewer active I/O ports thandoes CPU2 which has two active I/O ports, P2, P3 as a result of portrotation turn PR0. Hence during the course of the next port rotationPR1, I/O port rotation (block 440, FIG. 6) will be bypassed for CPU0,CPU1 and CPU3 as those CPUs execute task 400 following port rotationPR0. However, CPU0 executing the task 400 will set (block 455) the forcerotation flag to the other CPUs to initiate port rotation for the otherCPUs since it was previously determined (block 420) that it was time fora port rotation to occur. Also, CPU1 and CPU3 executing the task 400will each clear (block 450) that flag set for each since it waspreviously determined that I/O port rotation would be bypassed for CPU1and CPU3. This completes (block 412) the task 400 for CPU0, CPU1, andCPU3 over the course of port rotation PR1.

Conversely, the CPU2 executing the task 400 at the beginning of portrotation PR1, the next port rotation following port rotation PR0,determines (block 435) that CPU2 does not have fewer active I/O portsthan any other CPU (i.e. CPU2 has the same or more I/O ports that anyother CPU as a result of port rotation turn PR0). Hence, CPU2 willproceed to transfer ownership (block 440) of the longest owned activeI/O port (i.e. I/O port P2) from CPUn (i.e. CPU2) to CPUn+1 (i.e., CPU3)during the course of port rotation turn PR1 as shown in FIG. 5. Inaddition, since CPU2 is not (block 445) CPU0, and since CPU2 hascompleted its I/O port transfer, CPU2 will clear (block 450) the ForceRotation Flag stored in element 226 (FIG. 3b ) of the CPU shared datastructure 224 assigned to CPU2, completing (block 412) the task 400 forCPU2. Thus, the force rotation flag is set by the CPU0 for the otherCPUs and cleared by the other CPUs with every port rotation PR0, PR1 . .. PRn.

Similarly, the CPU3 executing the task 400 following the port rotationPR1 determines (block 422) that the force rotation flag has been set (byCPU0) for CPU3, and determines (block 435) that CPU3 does not have feweractive I/O ports than any other CPU (i.e. CPU3 has the same or more I/Oports that any other CPU following port rotation PR1. Hence, CPU3 willproceed to transfer ownership (block 440) of the longest owned activeI/O port (i.e. I/O port P4) from CPUn (i.e. CPU3) to CPU0 during thecourse of port rotation turn PR2 as shown in FIG. 5. In addition, sinceCPU3 is not (block 445) CPU0, CPU3 will clear (block 450) the ForceRotation Flag stored in element 226 (FIG. 3b ) of the CPU shared datastructure 224 assigned to CPU3, completing (block 412) the task 400 forCPU3.

CPU0 executing the task 400 following port rotation PR2 determines(block 435) that CPU0 does not have fewer active I/O ports than anyother CPU (i.e. CPU0 has the same or more I/O ports that any other CPUfollowing port rotation PR2). Hence, CPU0 will proceed to transferownership (block 440) of the longest owned active I/O port (i.e. I/Oport P0) from CPUn (i.e. CPU0) to CPUn+1 (i.e., CPU1) during the courseof port rotation PR3 as shown in FIG. 5. In addition, since CPU0 is(block 445) CPU0, CPU0 will set (block 455) the Force Rotation Flagstored in element 226 (FIG. 3b ) of the CPU shared data structure 224assigned to each of the other CPUs, CPU1, CPU2, and CPU3, to initiateport rotations for each of the other CPUs if appropriate as discussedabove, completing (block 412) the task for 400 for CPU0.

Thus, in the next port rotation PR4, following port rotation PR3, CPU1will respond (block 422) to the Force Rotation Flag previously set byCPU0 in port rotation turn PR4, and transfer (block 440) its longestowned I/O port (i.e. port P1) to the next CPU (i.e., CPU2), and thenclear (block 450) its Force Rotation Flag. In the next port rotation,i.e. port rotation turn PR5, CPU2 will respond (block 422) to the ForceRotation Flag previously set by CPU0 in port rotation turn PR5, andtransfer (block 440) its longest owned I/O port (i.e. port P3) to thenext CPU (i.e., CPU3), and then clear (block 450) its Force RotationFlag.

Operations will continue in this manner, one CPU such as CPU0 settingForce Rotation Flags for each of the other CPUs which in the same orsucceeding port rotations of the cycle, will rotate zero, one or moreI/O ports as appropriate until each CPU has rotated at least one I/Oport over the course of the cycle. The CPU0 will then, after expirationof the timer, again set the Force Rotation Flags to initiate anotherport rotation of another cycle of port rotations for each of the otherCPUs.

As can be seen in the example rotations above, every port willeventually be assigned to each CPU of the I/O adapter. Thus, in theembodiment of FIG. 5, the five active I/O ports are sequentially rotatedin the sequence P2, P4, P0, P1, P3 over the course of five portrotations PR1, PR2, PR3, PR4, PR5, respectively. The five active I/Oports are then sequentially rotated again in the sequence P2, P4, P0,P1, P3 over the course of next five port rotations. It is believed thatsuch an arrangement can reduce a penalty of an active port sharing a CPUwith another active port, by rotating the ports among the CPUs. It isfurther believed that when the workloads assigned the I/O ports lastlong enough, the amount of work performed for all ports in the I/Oadapter can be more even for all ports. Thus, in a Fibre Channel adapterin a storage controller, for example, it is believed that rotating theports once per second, for example, is sufficient to improve the balanceof the work performed for all active ports when measured once perminute. It is appreciated that other durations may be selected toinitiate a cycle of port rotations, depending upon the particularapplication.

FIG. 7 depicts another example of I/O ports being assigned and thenreassigned to a plurality of CPUs in a sequence of port rotations PR10,PR11, PR12 . . . PRn. In this example, the I/O adapter again has fourCPUs designated CPU0, CPU1, CPU2 and CPU3. In addition, the I/O adapterhas eight I/O ports P0-P7, in which all of the I/O ports P0-P7 areactively running I/O tasks such that no ports are idle. The I/O portsP0-P7 are evenly distributed such that each CPU has two active ports.

In accordance with the present description, it is believed that I/O portrotation may be beneficial in such instances in which the number ofactive I/O ports is evenly distributable among all of the CPUs, and eachCPU owns more than one port, since it is still possible that the workperformed for each port may be skewed. For example, ports owned by oneCPU may be performing just enough work to be considered active, whilethe ports owned by another CPU may be heavily stressed.

In such situations, it is recognized that a more balanced distributionof I/O ports may include one light port and one heavy port for each CPU.Such a distribution may be obtained by rotating the busy ports, evenwhen the number of active ports owned by each CPU is equal. However, insome applications, it may not be known by each CPU how busy its ownports are relative to the other ports in the adapter. However, it isbelieved that port rotations in accordance with the present descriptionmay nonetheless improve workload balance even if the degree of stress ona port relative to the other ports of the adapter is unknown by the CPUsince port rotation may provide a more balanced workload distribution atleast a portion of the time the workload is being performed.

Referring again to FIG. 6, a CPU executing the task 400 of FIG. 6 withthe I/O port distribution of FIG. 7, determines (block 410) whether thenumber of CPUs in the

I/O adapter is greater than or equal to the number of I/O ports in theI/O adapter. As set forth above, in the I/O port distribution of FIG. 7,there are eight active I/O ports distributed among four CPUs.Accordingly, I/O port rotation will not be bypassed as a result of thetest of operation 410.

Since the number of CPUs of the adapter is less than the number ofactive I/O ports, a determination (block 415) is made as to whether theCPU executing the task 400 is CPU0. As previously mentioned, in theillustrated embodiment, the CPU0 is selected to determine (block 420)whether to initiate I/O port rotations. In addition, as explained ingreater detail below, the CPU0 sets a flag for each of the other CPUs,CPU1, CPU2, CPU3, to initiate an I/O port rotation for those other CPUsif appropriate.

Again, in the illustrated embodiment each port rotation cycle may beinitiated upon expiration of a timer, for example. Should CPU0 determine(block 420) that the timer has not expired, the port rotation may bebypassed (block 412) until the timer expires.

In the embodiment of FIG. 7, each CPU of the adapter has two active I/Oports. Hence, in the embodiment of FIG. 7, no CPU will bypass I/O portrotation on account of not having any active ports (block 425).Moreover, in the embodiment of FIG. 7, all CPUs have the same number ofI/O ports, that is, two I/O ports. Hence, in the embodiment of FIG. 7,no CPU will bypass I/O port rotation on account of not having aninsufficient number of active ports (block 435). As a consequence, inthe embodiment of FIG. 7, port rotation will occur for all CPUs of theI/O adapter each time the force rotation flag is determined (block 422)to be set. As a result, in the embodiment of FIG. 7, each cycle of portrotations has a single port rotation such as port rotation PR11, forexample, such that each and every CPU of the I/O adapter rotatesownership of an I/O adapter to another CPU of the I/O adapter over thecourse of the single port rotation PR11. Similarly, each and every CPUof the I/O adapter rotates ownership of an I/O adapter to another CPU ofthe I/O adapter over the course of the single port rotation PR12, and soon.

Thus, for example, following port rotation PR10 and upon the nextexpiration of the timer, the next port rotation PR11 begins. In thecourse of the port rotation PR11, the CPU0 rotates (block 440) ownershipof its longest owned I/O port, that is port P0 in the example of FIG. 7)to the next CPU in sequence, that is, CPU1. In addition, the CPU0 sets(block 455) the force rotation flag for the other CPUs to initiate portrotation for the other CPUs over the course of port rotation PR11.

As each of the other CPUs determines (block 422) that the force rotationflag has been set for that CPU, each CPU rotates (block 440) ownershipof its longest owned I/O port, to the next CPU in sequence. Thus, asshown in FIG. 7, once the cycle of port rotation PR11 is completed, CPU1will have rotated its I/O port P2 (its longest owned I/O port) to CPU2(the next in sequence CPU), CPU2 will have rotated its I/O port P4 (itslongest owned I/O port) to CPU3 (the next in sequence CPU), and CPU3will have rotated its I/O port P6 (its longest owned I/O port) to CPU0(the next in sequence CPU).

Thus, in port rotation PR11, I/O ports P0, P2, P4 and P6 are rotated tothe next in sequence CPU. In the next port rotation PR12, I/O ports P1,P3, P5, and P7 are rotated to the next in sequence CPU. In the next portrotation PR13, I/O ports P0, P2, P4 and P6 are again rotated to the nextin sequence CPU. In the next port rotation PR14, I/O ports P1, P3, P5,and P7 are again rotated to the next in sequence CPU, and so on.

Accordingly, in each port rotation PR10, PR 11 . . . PRn, the ownershipof four of the eight active I/O ports will be rotated to the next CPU insequence. In the next succeeding port rotation, the remaining four ofthe eight active I/O ports will be rotated to the next CPU in sequence,and so on. As a consequence, it is believed that workload distributionmay be improved. For example, as a result of port rotation PR10, CPU0owns I/O ports P0, P1, as a result of port rotation PR11, CPU0 owns I/Oports P1, P6, as a result of port rotation PR12, CPU0 owns I/O ports P6,P7, and so on. Thus, the mix of I/O ports owned by a particular CPUchanges each port rotation cycle, which in this embodiment, includes asingle port rotation.

It is believed that such an arrangement can reduce a penalty of a veryactive port sharing a CPU with another very active port, by rotating theports among the CPUs. It is further believed that when the workloadsassigned the I/O ports last long enough, the amount of work performedfor all ports in the I/O adapter can be more even for all ports. Thus,in a Fibre Channel adapter in a storage controller, for example, it isbelieved that rotating the ports once per second is sufficient toimprove the balance of the work performed for all active ports whenmeasured once per minute. It is appreciated that other durations may beselected to initiate a cycle of port rotations, depending upon theparticular application.

Additional Embodiment Details

The described operations may be implemented as a method, apparatus orcomputer program product using standard programming and/or engineeringtechniques to produce software, firmware, hardware, or any combinationthereof. Accordingly, aspects of the embodiments may take the form of anentirely hardware embodiment, an entirely software embodiment (includingfirmware, resident software, micro-code, etc.) or an embodimentcombining software and hardware aspects that may all generally bereferred to herein as a “circuit,” “module” or “system.” Furthermore,aspects of the embodiments may take the form of a computer programproduct embodied in one or more computer readable medium(s) havingcomputer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, interconnected storage devices, anarray of storage devices, multiple memory or storage devices or anysuitable combination of the foregoing. In the context of this document,a computer readable storage medium may be any tangible medium that cancontain, or store a program for use by or in connection with aninstruction execution system, apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing.

Computer program code for carrying out operations for aspects of thepresent invention may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Smalltalk, C++ or the like and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Aspects of the present invention are described below with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

In certain embodiments, the system of FIG. 2 may be implemented as acloud component part in a cloud computing environment. In the cloudcomputing environment, the systems architecture of the hardware andsoftware components involved in the delivery of cloud computing maycomprise a plurality of cloud components communicating with each otherover a network, such as the Internet. For example, in certainembodiments, the system of FIG. 1 may provide clients, and other serversand software and/or hardware components in the networked cloud, withscheduling services.

The terms “an embodiment”, “embodiment”, “embodiments”, “theembodiment”, “the embodiments”, “one or more embodiments”, “someembodiments”, and “one embodiment” mean “one or more (but not all)embodiments of the present invention(s)” unless expressly specifiedotherwise.

The terms “including”, “comprising”, “having” and variations thereofmean “including but not limited to”, unless expressly specifiedotherwise.

The enumerated listing of items does not imply that any or all of theitems are mutually exclusive, unless expressly specified otherwise.

The terms “a”, “an” and “the” mean “one or more”, unless expresslyspecified otherwise.

Devices that are in communication with each other need not be incontinuous communication with each other, unless expressly specifiedotherwise. In addition, devices that are in communication with eachother may communicate directly or indirectly through one or moreintermediaries.

A description of an embodiment with several components in communicationwith each other does not imply that all such components are required. Onthe contrary a variety of optional components are described toillustrate the wide variety of possible embodiments of the presentinvention.

Further, although process steps, method steps, algorithms or the likemay be described in a sequential order, such processes, methods andalgorithms may be configured to work in alternate orders. In otherwords, any sequence or order of steps that may be described does notnecessarily indicate a requirement that the steps be performed in thatorder. The steps of processes described herein may be performed in anyorder practical. Further, some steps may be performed simultaneously.

When a single device or article is described herein, it will be readilyapparent that more than one device/article (whether or not theycooperate) may be used in place of a single device/article. Similarly,where more than one device or article is described herein (whether ornot they cooperate), it will be readily apparent that a singledevice/article may be used in place of the more than one device orarticle or a different number of devices/articles may be used instead ofthe shown number of devices or programs. The functionality and/or thefeatures of a device may be alternatively embodied by one or more otherdevices which are not explicitly described as having suchfunctionality/features. Thus, other embodiments of the present inventionneed not include the device itself.

The illustrated operations of FIGS. 4-7 shows certain events occurringin a certain order. In alternative embodiments, certain operations maybe performed in a different order, modified or removed. Moreover, stepsmay be added to the above described logic and still conform to thedescribed embodiments. Further, operations described herein may occursequentially or certain operations may be processed in parallel. Yetfurther, operations may be performed by a single processing unit or bydistributed processing units.

The foregoing description of various embodiments of the invention hasbeen presented for the purposes of illustration and description. It isnot intended to be exhaustive or to limit the invention to the preciseform disclosed. Many modifications and variations are possible in lightof the above teaching. It is intended that the scope of the invention belimited not by this detailed description, but rather by the claimsappended hereto. The above specification, examples and data provide acomplete description of the manufacture and use of the composition ofthe invention. Since many embodiments of the invention can be madewithout departing from the spirit and scope of the invention, theinvention resides in the claims hereinafter appended.

What is claimed is:
 1. A method, comprising: assigning input/output(I/O) ports to a plurality of central processing units (CPUs) of an I/Oadapter; for each CPU, transferring data between the CPU and each I/Oport assigned to that CPU; determining whether to initiate a cycle ofI/O port rotations; and upon determining to initiate a cycle of I/O portrotations, initiating a cycle of port rotations in which each portrotation of the cycle includes rotating an assignment of at least oneI/O port from one CPU to a different CPU of the plurality of CPUs andwherein an I/O port assignment for each CPU of the plurality CPUs isrotated for at least a portion of the cycle and wherein each portrotation of said cycle of I/O port rotations includes determining foreach CPU, whether to rotate a port assignment of the particular CPU to adifferent CPU of the plurality of CPUs, and if so, selecting the activeI/O port which has been owned the longest by the particular CPU.
 2. Themethod of claim 1 wherein the data transferring includes initiating atask of transfer of data at said at least one I/O port under control ofsaid one CPU and continuing the task of transfer of data at said atleast one I/O port under control of said different CPU following therotation of assignment of said at least one I/O port from said one CPUto said different CPU.
 3. The method of claim 1 wherein said determiningwhether to initiate a cycle of I/O port rotations, includes determiningwhether a predetermined time period has expired.
 4. The method of claim1 wherein the initiating a cycle of I/O port rotations includes settingby one CPU, a rotation flag for the remaining CPUs of the plurality ofCPUs, to initiate port rotations for the remaining CPUs of the pluralityof CPUs.
 5. The method of claim 1 wherein each port rotation of saidcycle of I/O port rotations includes determining for each CPU, whetherto rotate a port assignment of the particular CPU to a different CPU ofthe plurality of CPUs.
 6. The method of claim 5 wherein the initiating acycle of I/O port rotations includes setting by one CPU, a rotation flagfor the remaining CPUs of the plurality of CPUs, to initiate portrotations for the remaining CPUs of the plurality of CPUs, and whereinthe determining for each CPU whether to rotate a port assignment,includes checking whether the rotation flag has been set for thatparticular CPU.
 7. The method of claim 5 wherein the determining foreach CPU whether to rotate a port assignment, includes determiningwhether that particular CPU has any active I/O ports, and if so, whetherthat CPU has fewer active I/O ports than any other CPU of the pluralityof CPUs.
 8. The method of claim 1 wherein the CPUs of the plurality ofCPUs are sequentially numbered in a sequence and wherein the rotating anassignment of at least one I/O port from one CPU to a different CPU ofthe plurality of CPUs includes rotating an assignment of at least oneI/O port from one CPU to the next sequentially numbered CPU in thesequence of the sequentially numbered CPUs.
 9. A computer programproduct for use with an I/O adapter having a plurality of centralprocessing units (CPUs) and a plurality of input/output (I/O) portscoupled to at least one of hosts and storage in a storage area network(SAN), the computer program product comprising a computer readablestorage device having computer readable program code embodied thereinthat executes to perform operations, the operations comprising:assigning input/output (I/O) ports to a plurality of central processingunits (CPUs) of an I/O adapter; for each CPU, transferring data betweenthe CPU and each I/O port assigned to that CPU; determining whether toinitiate a cycle of I/O port rotations; and upon determining to initiatea cycle of I/O port rotations, initiating a cycle of port rotations inwhich each port rotation of the cycle includes rotating an assignment ofat least one I/O port from one CPU to a different CPU of the pluralityof CPUs and wherein an I/O port assignment for each CPU of the pluralityCPUs is rotated for at least a portion of the cycle and wherein eachport rotation of said cycle of I/O port rotations includes determiningfor each CPU, whether to rotate a port assignment of the particular CPUto a different CPU of the plurality of CPUs, and if so, selecting theactive I/O port which has been owned the longest by the particular CPU.10. The computer program product of claim 9 wherein the datatransferring includes initiating a task of transfer of data at said atleast one I/O port under control of said one CPU and continuing the taskof transfer of data at said at least one I/O port under control of saiddifferent CPU following the rotation of assignment of said at least oneI/O port from said one CPU to said different CPU.
 11. The computerprogram product of claim 9 wherein said determining whether to initiatea cycle of I/O port rotations, includes determining whether apredetermined time period has expired.
 12. The computer program productof claim 9 wherein the initiating a cycle of I/O port rotations includessetting by one CPU, a rotation flag for the remaining CPUs of theplurality of CPUs, to initiate port rotations for the remaining CPUs ofthe plurality of CPUs.
 13. The computer program product of claim 9wherein each port rotation of said cycle of I/O port rotations includesdetermining for each CPU, whether to rotate a port assignment of theparticular CPU to a different CPU of the plurality of CPUs.
 14. Thecomputer program product of claim 13 wherein the initiating a cycle ofI/O port rotations includes setting by one CPU, a rotation flag for theremaining CPUs of the plurality of CPUs, to initiate port rotations forthe remaining CPUs of the plurality of CPUs, and wherein the determiningfor each CPU whether to rotate a port assignment, includes checkingwhether the rotation flag has been set for that particular CPU.
 15. Thecomputer program product of claim 13 wherein the determining for eachCPU whether to rotate a port assignment, includes determining whetherthat particular CPU has any active I/O ports, and if so, whether thatCPU has fewer active I/O ports than any other CPU of the plurality ofCPUs.
 16. The computer program product of claim 9 wherein the CPUs ofthe plurality of CPUs are sequentially numbered in a sequence andwherein the rotating an assignment of at least one I/O port from one CPUto a different CPU of the plurality of CPUs includes rotating anassignment of at least one I/O port from one CPU to the nextsequentially numbered CPU in the sequence of the sequentially numberedCPUs.
 17. A system for use with a host, storage and a storage areanetwork (SAN), comprising: an I/O adapter having a plurality ofprocessors including a plurality of central processing units (CPUs),aplurality of input/output (I/O) ports coupled to at least one of hostsand storage in a storage area network (SAN), and a non-transitorycomputer readable storage medium including code executed by a processorto perform operations, the operations comprising: assigning input/output(I/O) ports to a plurality of central processing units (CPUs) of the I/Oadapter; for each CPU, transferring data between the CPU and each I/Oport assigned to that CPU; determining whether to initiate a cycle ofI/O port rotations; and upon determining to initiate a cycle of I/O portrotations, initiating a cycle of port rotations in which each portrotation of the cycle includes rotating an assignment of at least oneI/O port from one CPU to a different CPU of the plurality of CPUs andwherein an I/O port assignment for each CPU of the plurality CPUs isrotated for at least a portion of the cycle and wherein each portrotation of said cycle of I/O port rotations includes determining foreach CPU, whether to rotate a port assignment of the particular CPU to adifferent CPU of the plurality of CPUs, and if so, selecting the activeI/O port which has been owned the longest by the particular CPU.
 18. Thesystem of claim 17 wherein the data transferring includes initiating atask of transfer of data at said at least one I/O port under control ofsaid one CPU and continuing the task of transfer of data at said atleast one I/O port under control of said different CPU following therotation of assignment of said at least one I/O port from said one CPUto said different CPU.
 19. The system of claim 17 wherein saiddetermining whether to initiate a cycle of I/O port rotations, includesdetermining whether a predetermined time period has expired.
 20. Thesystem of claim 17 wherein the initiating a cycle of I/O port rotationsincludes setting by one CPU, a rotation flag for the remaining CPUs ofthe plurality of CPUs, to initiate port rotations for the remaining CPUsof the plurality of CPUs.
 21. The system of claim 17 wherein each portrotation of said cycle of I/O port rotations includes determining foreach CPU, whether to rotate a port assignment of the particular CPU to adifferent CPU of the plurality of CPUs.
 22. The system of claim 21wherein the initiating a cycle of I/O port rotations includes setting byone CPU, a rotation flag for the remaining CPUs of the plurality ofCPUs, to initiate port rotations for the remaining CPUs of the pluralityof CPUs, and wherein the determining for each CPU whether to rotate aport assignment, includes checking whether the rotation flag has beenset for that particular CPU.
 23. The system of claim 21 wherein thedetermining for each CPU whether to rotate a port assignment, includesdetermining whether that particular CPU has any active I/O ports, and ifso, whether that CPU has fewer active I/O ports than any other CPU ofthe plurality of CPUs.
 24. The system of claim 17 wherein the CPUs ofthe plurality of CPUs are sequentially numbered in a sequence andwherein the rotating an assignment of at least one I/O port from one CPUto a different CPU of the plurality of CPUs includes rotating anassignment of at least one I/O port from one CPU to the nextsequentially numbered CPU in the sequence of the sequentially numberedCPUs.